Adaptive Critics for Airline Revenue Management

number: 007-0058 Adaptive Critics for Airline Revenue Management Abhijit Gosavi Department of Industrial and Systems Engineering University at Buffalo, SUNY 317 Bell Hall, Buffalo, NY 14260 [email protected] POMS 18th Annual Conference Dallas, Texas, U.S.A. May 4 to May 7, 2007. Abstract We present an approximate dynamic programming (DP) technique, called adaptive critic, for solving an airline revenue management problem. The revenue management problem is cast as a semi-Markov decision problem (semi-MDP). Werbos [25] first suggested the use of policy iteration in approximate DP, and the adaptive critic essentially falls within this class of algorithms. Barto et al [1] wrote a paper that provided a stepsize-based technique for this algorithm. Recently, Konda and Borkar [15] analyzed a modified version for its convergence properties. All three of these works focus on the MDP. We combine notions from these works to solve the associated semi-MDP and prove its convergence. We provide computational evidence to demonstrate the efficacy of our method. In addition, we extend our technique to handle Markowitz risk, which should appeal to risk-sensitive airline managers seeking to avoid bankruptcy.We present an approximate dynamic programming (DP) technique, called adaptive critic, for solving an airline revenue management problem. The revenue management problem is cast as a semi-Markov decision problem (semi-MDP). Werbos [25] first suggested the use of policy iteration in approximate DP, and the adaptive critic essentially falls within this class of algorithms. Barto et al [1] wrote a paper that provided a stepsize-based technique for this algorithm. Recently, Konda and Borkar [15] analyzed a modified version for its convergence properties. All three of these works focus on the MDP. We combine notions from these works to solve the associated semi-MDP and prove its convergence. We provide computational evidence to demonstrate the efficacy of our method. In addition, we extend our technique to handle Markowitz risk, which should appeal to risk-sensitive airline managers seeking to avoid bankruptcy.